In the case where plural distributed programs, which have been independently developed, are executed in combination, systems such as MapReduce described, for example, in Non-Patent Document 1 have been used to execute these distributed programs. In order to combine and execute plural programs written in MapReduce, it is necessary to use files for transmitting and receiving data between these programs. More specifically, the results of execution of one program are written out in a form of file, and through the file, the results of the execution of the program are inputted into another program.
However, using the file for transmitting and receiving the data between the programs causes a problem of efficiency. In general, the next program starts to be executed after the preceding program completely ends. However, it is possible to execute these programs in parallel by starting the next program to execute at a point in time when the preceding program is partially outputted.
In connection with sequential programs, a concept called “pipe” has been known. In the case where the pipe is used, the output from one program can be used as input of another program through a memory, whereby it is possible to start to execute the next program without waiting for the preceding program to completely end. For this reason, this pipe can be used for the distributed program, and it is possible to combine and execute plural distributed programs using this pipe.
On the other hand, in order to realize the distributed program, it is necessary to enhance reliability of the system. This is because, in the case where programs are executed on a large number of servers, the probability of failure to occur in one server is larger, as compared with the case where programs are sequentially executed on a single server.
MapReduce achieves improved reliability on the assumption that input and output between the programs are performed through highly reliable file systems. Thus, it cannot achieve the high reliability in the case where the pipe is used for input and output between the programs.